Skip to content

test(profiling): unflake test_copy_memory_stats - #19706

Open
KowalskiThomas wants to merge 2 commits into
mainfrom
kowalski/test-profiling-unflake-test_copy_memory_stats
Open

test(profiling): unflake test_copy_memory_stats#19706
KowalskiThomas wants to merge 2 commits into
mainfrom
kowalski/test-profiling-unflake-test_copy_memory_stats

Conversation

@KowalskiThomas

@KowalskiThomas KowalskiThomas commented Aug 14, 2026

Copy link
Copy Markdown
Collaborator

Description

This PR attempts to unflake test_copy_memory_stats which has been rarely (but still) failing lately. I suspect this is because there is a race between the sampling thread upgrading to fast copy memory and the uploader thread dumping stats to the file (which would explain why it's so rare).

Something LLMs also suggested is to make sure we sort file names by their actual order and not just with sorted, since sorted would sort lexicographically in which case profile.10 would appear before profile.2, so I also fixed that.

@KowalskiThomas KowalskiThomas added changelog/no-changelog A changelog entry is not required for this PR. Profiling Continous Profling labels Aug 14, 2026
@datadog-prod-us1-3

datadog-prod-us1-3 Bot commented Aug 14, 2026 •
edited by datadog-official Bot
Loading

Copy link
Copy Markdown

Tests

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 41b6359 | Docs | Datadog PR Page | Give us feedback!

@cit-pr-commenter-54b7da

cit-pr-commenter-54b7da Bot commented Aug 14, 2026

Copy link
Copy Markdown

Codeowners resolved as

Resolved from the full PR diff against main using the target branch CODEOWNERS file.
CODEOWNERS team requests not listed below are not required by the current file set.

tests/profiling/collector/pprof_utils.py                                @DataDog/profiling-python
tests/profiling/collector/test_asyncio_task_count.py                    @DataDog/profiling-python
tests/profiling/collector/test_copy_memory_stats.py                     @DataDog/profiling-python
tests/profiling/collector/test_greenlet_count.py                        @DataDog/profiling-python
tests/profiling/collector/test_heap_tracker_count.py                    @DataDog/profiling-python
tests/profiling/collector/test_internal_adaptive_sampling.py            @DataDog/profiling-python
tests/profiling/collector/test_sample_count.py                          @DataDog/profiling-python
tests/profiling/collector/test_thread_subsampling.py                    @DataDog/profiling-python

@cit-pr-commenter-54b7da

cit-pr-commenter-54b7da Bot commented Aug 14, 2026

Copy link
Copy Markdown

Dependency direction analysis

⚠️ Existing dependency direction violations

There are 255 dependency direction violations that already exist on the base branch and have not been changed by this PR.

Show existing violations (showing 5 of 255 highest severity) 
ddtrace.internal.tracemethods -×-> ddtrace.trace  (internal-core -> product:tracing, score=134)
ddtrace.llmobs._evaluators.runner -×-> ddtrace.trace  (product:llmobs -> product:tracing, score=132)
ddtrace.llmobs._integrations.openai -×-> ddtrace.trace  (product:llmobs -> product:tracing, score=132)
ddtrace.appsec._contrib.django -×-> ddtrace.trace  (product:appsec -> product:tracing, score=132)
ddtrace.llmobs._integrations.anthropic -×-> ddtrace.trace  (product:llmobs -> product:tracing, score=132)

To see all violations, download the layers-base.json and layers-pr.json artifacts from this CI job and run:

uv run --script scripts/import-analysis/layers.py compare layers-base.json layers-pr.json

@cit-pr-commenter-54b7da

cit-pr-commenter-54b7da Bot commented Aug 14, 2026

Copy link
Copy Markdown

Circular import analysis

⚠️ Existing circular imports

There are 5 circular imports that already exist on the base branch and have not been changed by this PR.

ddtrace.contrib.internal.django.patch -> ddtrace.contrib.internal.django.response -> ddtrace.contrib.internal.django.patch
ddtrace.contrib.internal.pytorch._distributed -> ddtrace.contrib.internal.pytorch._rank_root -> ddtrace.contrib.internal.pytorch._distributed
ddtrace.llmobs -> ddtrace.llmobs._evaluators -> ddtrace.llmobs._evaluators.format -> ddtrace.llmobs._experiment -> ddtrace.llmobs
ddtrace.errortracking._handled_exceptions.bytecode_injector -> ddtrace.errortracking._handled_exceptions.callbacks -> ddtrace.errortracking._handled_exceptions.collector -> ddtrace.errortracking._handled_exceptions.bytecode_reporting -> ddtrace.errortracking._handled_exceptions.bytecode_injector
ddtrace.appsec._asm_request_context -> ddtrace.appsec._iast._iast_request_context_base -> ddtrace.appsec._iast._iast_env -> ddtrace.appsec._iast.reporter -> ddtrace.appsec._exploit_prevention.stack_traces -> ddtrace.appsec._asm_request_context

@KowalskiThomas
KowalskiThomas marked this pull request as ready for review August 14, 2026 15:02
@KowalskiThomas
KowalskiThomas requested a review from a team as a code owner August 14, 2026 15:02
taegyunkim
taegyunkim approved these changes Aug 14, 2026
View reviewed changes

@taegyunkim taegyunkim left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I remember that we had to properly sort the pprof files as well.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@taegyunkim taegyunkim taegyunkim approved these changes
@vlad-scherbich vlad-scherbich Awaiting requested review from vlad-scherbich vlad-scherbich is a code owner automatically assigned from DataDog/profiling-python

Assignees

No one assigned

Labels

changelog/no-changelog A changelog entry is not required for this PR. Profiling Continous Profling

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@KowalskiThomas @taegyunkim